Encoder and Decoder and Methods Thereof for Encoding/Decoding a Picture of a Video Sequence

ABSTRACT

An object of the embodiments is to achieve an improved reference picture handling. That is achieved by taking into account whether the reference pictures in the decoded picture buffer are long-term reference pictures or short-term reference pictures when determining how they should be marked when the information of the reference picture set is received. The reference pictures are marked as “used for short-term reference” or “used for long-term reference” in the Decoded Picture Buffer (DPB) depending on whether they are included as short-term pictures or long-term pictures in the RPS of a current picture.

RELATED APPLICATION

This application is a continuation of U.S. application Ser. No.15/628,005 filed Jun. 20, 2017, which is a continuation of U.S.application Ser. No. 14/996,711 filed Jan. 15, 2016, now U.S. Pat. No.9,706,218, which is a continuation of U.S. application Ser. No.13/643,688 filed Nov. 16, 2012, now U.S. Pat. No. 9,277,219, which is anational stage application of Patent Cooperation Treaty Application No.PCT/SE2012/051140 filed Oct. 23, 2012, which claims the benefit of U.S.Provisional Application No. 61/550,646 filed Oct. 24, 2011, thedisclosures of each are incorporated herein by reference in theirentirety.

TECHNICAL FIELD

The embodiments of the present invention relates to a decoder and anencoder and methods thereof for managing reference pictures to be usedfor decoding an encoded representation of a picture of a video sequence.

BACKGROUND

High Efficiency Video Coding (HEVC) is a new video coding standardcurrently being developed in Joint Collaborative Team—Video Coding(JCT-VC). JCT-VC is a collaborative project between Moving PictureExperts Group (MPEG) and International TelecommunicationUnion—Telecommunication Standardization Sector (ITU-T). Currently, anHEVC Model (HM) is defined that includes a number of new tools and isconsiderably more efficient than H.264/Advanced Video Coding (AVC).

A picture in HEVC is partitioned into one or more slices, where eachslice is an independently decodable segment of the picture. This meansthat if a slice is missing, for instance got lost during transmission,the other slices of that picture can still be decoded correctly. Inorder to make slices independent, they do not depend on each other. Nobitstream element of another slice of the same picture is required fordecoding any element of a particular slice.

Each slice contains a slice header which independently provides allrequired data for the slice to be independently decodable. One exampleof a data element present in the slice header is the slice address,which is used for the decoder to know the spatial location of the slice.Another example is the Buffer Description also referred to as ReferencePicture Set which contains information of which reference pictures to beused when decoding a picture. However, these are only examples of dataelements in the slice header.

HEVC has mechanisms for handling reference pictures. The referencepictures are previously decoded pictures to be used for decoding of acurrent picture. A decoded picture buffer (DPB) contains picturesdecoded by the decoder. A reference picture in HEVC is a picture in thedecoded picture buffer (DPB) that is available for reference by beingmarked “used for reference”. There may also be pictures in the DPB thatare marked “unused for reference”, these pictures are not available forreference and are not reference pictures.

HEVC uses absolute signaling of reference pictures. The absolutesignaling is realized by signaling what reference pictures to keep atthe decoder. The signaling is done in a Buffer Description also referredto as Reference Picture Set (RPS), for each picture explicitly or byusing a reference to a Sequence Parameter Set (SPS). The RPS alsocontains an indication of which pictures can be used for reference bythe current picture. Reference pictures indicated to be used by thecurrent picture are included in reference picture lists in the decoder.The reference picture lists are then used in the decoding process of thecurrent slice of the current picture.

Each reference picture in the RPS is either identified as a short-termpicture or as a long-term picture. The information if it is a short-termor a long-term reference picture is signaled in the RPS by sending twoseparate sets, one with all short-term reference pictures and one withall long-term reference pictures. An alternative design would be to senda single list and for each element indicate with a flag if it is along-term reference picture or a short-term reference picture.

Picture Order Count (POC) is used in HEVC to define the output order (ordisplay order) of pictures and also to identify reference pictures.Syntax elements used to derive the POC is signaled for each referencepicture in the RPS. For short-term reference pictures the values of POCin the RPS must be identical to the values of POC signaled in the sliceheader of the reference picture to which the values of the POC in theRPS are referring. For long-term reference pictures there are twooptions; either the values of POC in the RPS are identical to the valuesof POC signaled in the slice header of the reference picture to whichthe values of the POC in the RPS are referring or the values of POC inthe RPS are a shorter representation of to the values of POC signaled inthe slice header (a.k.a. the least significant bits of the picture ordercount value or the POC_LSB) of the reference picture to which the valuesof the POC in the RPS are referring such that the reference pictures areuniquely identifiable. The latter is only allowed when there is only onepicture in the DPB with a specific POC_LSB. For long-term referencepictures the term “corresponds to” is used to denote the identificationof a reference picture using any of the two above mentioned options.

When buffer descriptions were originally proposed for inclusion in HEVC,the proposal included a marking process performed so that referencepictures that are in the DPB but not included in the RPS are marked as“unused for reference” prior to decoding of the current picture. Theoutput process is also performed prior to the decoding of the currentpicture.

SUMMARY

An object of the embodiments is to achieve an improved reference picturehandling. That is achieved by taking into account whether the referencepictures in the decoded picture buffer are long-term reference picturesor short-term reference pictures when determining how they should bemarked when the information of the reference picture set is received.The reference pictures are marked as “used for short-term reference” or“used for long-term reference” in the Decoded Picture Buffer (DPB)depending on whether they are included as short-term pictures orlong-term pictures in the RPS of a current picture.

According to a first aspect according to embodiments, a method performedby a decoder for managing reference pictures to be used for decoding anencoded representation of a picture of a video sequence is provided. Inthe method, a reference picture set received from an encoder is decoded,wherein the reference picture set comprises information of the referencepictures to be kept in a decoded picture buffer. A POC value indicatedin the reference picture set is decoded and it is determined if the POCvalue indicated in the reference picture set corresponds to a short-termreference picture or a long-term reference picture. If the POC valueindicated in the reference picture set corresponds to a long-termreference picture:

if there is a long-term reference picture in the decoded picture bufferwith a POC value corresponding to the POC value indicated in thereference picture set, keeping the long-term reference picture in adecoded picture buffer as a long-term reference picture,

if there is a short-term reference picture in the decoded picture bufferwith a POC value corresponding to the POC value indicated in thereference picture set, marking the short-term reference picture in thedecoded picture buffer as a long-term reference picture and keeping itin decoded picture buffer,

if the POC value indicated in the reference picture set corresponds to ashort-term reference picture:

if there is a short-term reference picture in the decoded picture bufferwith a POC value corresponding to the POC value indicated in thereference picture set, keeping the short-term reference picture in thedecoded picture buffer as a short-term reference picture.

According to a second aspect according to embodiments, a decoder formanaging reference pictures to be used for decoding an encodedrepresentation of a picture of a video sequence is provided. The decoderis configured to decode a reference picture set received from anencoder, wherein the reference picture set comprises information of thereference pictures to be kept in a decoded picture buffer and to decodea POC value indicated in the reference picture set. The decodercomprises a processor configured to determine if the POC value indicatedin the reference picture set corresponds to a short-term referencepicture or a long-term reference picture. Further, the processor isconfigured, when there is a long-term reference picture in the decodedpicture buffer with a POC value corresponding to the POC value indicatedin the reference picture set and when the POC value indicated in thereference picture set corresponds to a long-term reference picture, tokeep the long-term reference picture in a decoded picture buffer as along-term reference picture, the processor is further configured, whenthere is a short-term reference picture in the decoded picture bufferwith a POC value corresponding to the POC value indicated in thereference picture set and when the POC value indicated in the referencepicture set corresponds to a long-term reference picture, to mark theshort-term reference picture in the decoded picture buffer as along-term reference picture and keeping it in decoded picture buffer.The processor is further configured, when there is a short-termreference picture in the decoded picture buffer with a POC valuecorresponding to the POC value indicated in the reference picture setand when the POC value indicated in the reference picture setcorresponds to a short-term reference picture, to keep the short-termreference picture in the decoded picture buffer as a short-termreference picture.

According to a third aspect according to embodiments, a method performedby an encoder for managing reference pictures to be used for encoding anencoded representation of a picture of a video sequence is provided. Inthe method, a POC value is assigned wherein the POC value is selectedsuch that a decoder can perform the following steps:

decoding a reference picture set received from an encoder, wherein thereference picture set comprises information of the reference pictures tobe kept in a decoded picture buffer:

decoding a Picture Order Count, POC, value indicated in the referencepicture set and

determining if the POC value indicated in the reference picture setcorresponds to a short-term reference picture or a long-term referencepicture,

if the POC value indicated in the reference picture set corresponds to along-term reference picture:

if there is a long-term reference picture in the decoded picture bufferwith a POC value corresponding to the POC value indicated in thereference picture set, keeping the long-term reference picture in adecoded picture buffer as a long-term reference picture,

if there is a short-term reference picture in the decoded picture bufferwith a POC value corresponding to the POC value indicated in thereference picture set, marking the short-term reference picture in thedecoded picture buffer as a long-term reference picture and keeping itin decoded picture buffer,

if the POC value indicated in the reference picture set corresponds to ashort-term reference picture:

if there is a short-term reference picture in the decoded picture bufferwith a POC value corresponding to the POC value indicated in thereference picture set, keeping the short-term reference picture in thedecoded picture buffer as a short-term reference picture and

including the assigned POC value in a reference picture set to be sentto a decoder, wherein the reference picture set comprises information ofthe reference pictures to be kept in a decoded picture buffer of thedecoder.

According to a fourth aspect according to embodiments, an encoder formanaging reference pictures to be used for encoding an encodedrepresentation of a picture of a video sequence is provided. The encodercomprises a processor for assigning a POC value wherein the POC value isselected such that a decoder can perform the following steps

decoding a reference picture set received from an encoder, wherein thereference picture set comprises information of the reference pictures tobe kept in a decoded picture buffer:

decoding a POC value indicated in the reference picture set and

determining if the POC value indicated in the reference picture setcorresponds to a short-term reference picture or a long-term referencepicture,

if the POC value indicated in the reference picture set corresponds to along-term reference picture:

if there is a long-term reference picture in the decoded picture bufferwith a POC value corresponding to the POC value indicated in thereference picture set, keeping (206) the long-term reference picture ina decoded picture buffer as a long-term reference picture,

if there is a short-term reference picture in the decoded picture bufferwith a POC value corresponding to the POC value indicated in thereference picture set, marking (207) the short-term reference picture inthe decoded picture buffer as a long-term reference picture and keepingit in decoded picture buffer,

if the POC value indicated in the reference picture set corresponds to ashort-term reference picture:

if there is a short-term reference picture in the decoded picture bufferwith a POC value corresponding to the POC value indicated in thereference picture set, keeping (208) the short-term reference picture inthe decoded picture buffer as a short-term reference picture.

and the processor is further configured to include the assigned POCvalue in a reference picture set to be sent to a decoder, wherein thereference picture set comprises information of the reference pictures tobe kept in a decoded picture buffer of the decoder.

An advantage with the embodiments is that the taking into accountwhether the reference pictures in the decoded picture buffer arelong-term reference pictures or short-term reference when performing thepicture marking process improves possibilities for detecting erroneousbitstream which is thus useful for example for error detection in errorprone networks.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates schematically reference picture handling according toembodiments of the present invention.

FIG. 2 is a flowchart illustrating the method in a decoder according toan embodiment of the present invention.

FIG. 3 illustrates an encoder in the context of the embodiments of thepresent invention.

FIG. 4 illustrates a decoder according to the embodiments of the presentinvention.

FIG. 5 is a flowchart illustrating the method in an encoder according toan embodiment of the present invention.

DETAILED DESCRIPTION

With reference to FIG. 1, the encoder 300 informs the decoder 400 aboutwhich reference pictures to use for decoding of a particular picture byincluding the POCs of the reference pictures in the reference pictureset (RPS) 101. The RPS is sent for each picture either in the sliceheader for the particular picture or in other control information. Thedecoder 400 stores decoded pictures in a decoded picture buffer (DPB)103 and marks the pictures of the DPB as either used for reference or asnot used for reference. In HEVC, the marking is performed beforedecoding of a current picture.

The reference pictures are included in the RPS 101 either as short-term(st) reference pictures or long-term (It) reference pictures andaccording to the embodiments there is a restriction that pictures thatonce have been included in the RPS as long-term pictures cannot later beincluded as, or be converted to, short-term pictures in the DPB. Thus,the reference pictures are marked as “used for short-term reference” or“used for long-term reference” in the Decoded Picture Buffer (DPB)depending on whether they are included as short-term pictures orlong-term pictures in the RPS of a current picture.

The POCs of the respective pictures in the DPB marked as used forreference are inserted into reference picture lists 105 a, 105 b e.g.denoted RefPicSetLtCurr, RefPicSetLtFoll, RefPicSetStCurrBefore,RefPicSetStCurrAfter or RefPicSetStFoll. The POCs of the long-termreference pictures are inserted in RefPicSetLtCurr or RefPicSetLtFolland the POCs of the short-term reference pictures are inserted inRefPicSetStCurrBefore, RefPicSetStCurrAfter or RefPicSetStFoll.

From the POCs of the reference pictures in those lists 105 a, 105 b, thedecoder 400 constructs the final reference picture lists 107 whichcontains the POCs of the reference pictures that should be used by thedecoder for decoding. This procedure according to the embodiments isillustrated by the following example.

In addition, it should be noted that although the embodiments areexplained in the context of the decoder, the embodiments also apply tothe encoder, since the encoder mimics the decoder behavior to ensurethat decoded pictures are created exactly as they should. The encoderdecides which POC values to assign to each picture it encodes and itdecides which old pictures that it wants to keep as reference picturesand which pictures that should be short-term and which that should belong-term. While saying that, the encoder is bound to old decisions, ifa picture is marked as long-term earlier, it cannot be re-marked asshort-term, pictures that are marked unused for reference cannot be usedfor reference in any future picture and so on.

In the encoder the following steps are performed which is illustrated inthe flowchart of FIG. 5:

501. The encoder selects the POC for the current picture. Thus, theencoder selects what POC value to signal to the decoder to identify thecurrent picture. The encoder may select any POC value as long at is doesnot make the bitstream conflict with any bitstream requirement such asPOC shall represent output order. That implies that the POC value mustbe assigned by the encoder such that the decoder can behave according tothe embodiments of the present invention. In addition to the POC, theencoder also sends RPS syntax to the decoder to control what referencepictures to use and which ones should be short-term and which should belong-term.

502. The encoder encodes an RPS including the POCs of all short-termreference pictures and long-term reference pictures that are used by thecurrent picture or may be used by pictures following the currentpicture.

503. The picture is encoded using the reference pictures indicated bythe RPS.

Accordingly, an encoder for managing reference pictures to be used forencoding an encoded representation of a picture of a video sequence isprovided. The encoder comprises a processor for assigning a POC valuesuch that a decoder can perform the following steps

decoding 201 a reference picture set received from an encoder, whereinthe reference picture set comprises information of the referencepictures to be kept in a decoded picture buffer:

decoding 202 a Picture Order Count, POC, value indicated in thereference picture set and

determining 203 if the POC value indicated in the reference picture setcorresponds to a short-term reference picture or a long-term referencepicture,

if the POC value indicated in the reference picture set corresponds to along-term reference picture:

keeping 206 a long-term reference picture in a decoded picture buffer asa long-term reference picture if there is a long-term reference picturein the decoded picture buffer with a POC value corresponding to the POCvalue indicated in the reference picture set,

marking 207 a short-term reference picture in the decoded picture bufferas a long-term reference picture and keeping it in decoded picturebuffer if there is a short-term reference picture in the decoded picturebuffer with a POC value corresponding to the POC value indicated in thereference picture set,

if the POC value indicated in the reference picture set corresponds to ashort-term reference picture:

keeping 208 a short-term reference picture in the decoded picture bufferas a short-term reference picture if there is a short-term referencepicture in the decoded picture buffer with a POC value corresponding tothe POC value indicated in the reference picture set, and the processoris further configured to include the assigned POC value in a referencepicture set to be sent to a decoder, wherein the reference picture setcomprises information of the reference pictures to be kept in a decodedpicture buffer of the decoder.

In the decoder the following steps are performed according toembodiments of the present invention:

1. Turning now to the flowchart of FIG. 2, the decoder receives acurrent picture and the RPS for that picture e.g. in the slice headerand decodes the RPS 201 of the current picture before decoding thepicture. It is preferred that the long-term reference pictures in theDPB are handled before the short-term reference pictures to mark thereference pictures in the DPB in order to avoid that one referencepicture is included in the reference picture list as both a short-termand a long-term reference picture.

2. The POC of each reference picture of the RPS is decoded 202 and thefollowing steps are performed:

a) If the POC of the reference picture is included in the RPS as along-term reference picture 203 with POC y and there is a referencepicture with that POC y in the DPB marked as “used for long-termreference” 205 that picture is kept 206 in the DPB marked as “used forlong-term reference”.

b) If POC of the reference picture is in included in the RPS as along-term reference picture 203 and there is a picture with that POC inthe DPB marked as “used for short-term reference” 205 that picture iskept in the DPB but remarked as “used for long-term reference” 207.According to an embodiment, step 206 (keeping a long-term referencepicture in a decoded picture buffer as a long-term reference picture ifthere is a long-term reference picture in the decoded picture bufferwith a POC value equal to the POC value indicated in the referencepicture set) is performed before step 207 (marking a short-termreference picture in the decoded picture buffer as a long-term referencepicture and keeping it in decoded picture buffer if there is ashort-term reference picture in the decoded picture buffer with a POCvalue equal to the POC value indicated in the reference picture set).I.e. step 206 is performed and if the condition of step 206 is notfulfilled, step 207 is performed.

  i.e. if (there is a long-term reference picture picX in the DPB     with pic_order_cnt_isb equal to PocLtCurr[i])    RefPicSetLtCurr[i]= picX   else if (there is a short-term reference picture picY in theDPB      with pic_order_cnt_isb equal to PocLtCurr[i])   RefPicSetLtCurr[i] = picY

Wherein the pic_order_cnt_lsb is the POC value and PocLtCurr[i] is alist of long-term reference pictures in the RPS and theRefPicSetLtCurr[i] is a reference picture list containing long-termreference pictures.

c) Otherwise, if the POC of the reference picture is included in the RPSas a short-term reference picture 203 and if there is a picture withthat POC in the DPB marked as “used for short-term reference” thatpicture is kept 208 in the DPB marked as “used for short-termreference”.

  i.e., for(i = 0; i < NumPocStCurrBefore; i++ )    if( there is ashort-term reference picture picX in the DPB       with PicOrderCntValequal to PocStCurrBefore[i])      RefPicSetStCurrBefore[i] = picX   else    RefPicSetStCurrBefore[i] = “no reference picture”

Wherein the PicOrderCntVal is the POC value and PocStCurrBefore[i] is alist of long-term reference pictures in the RPS and theRefPicSetStCurrBefore[i] is a reference picture list containingshort-term reference pictures.

All reference pictures included in RefPicSetStCurrBefore,RefPicSetStCurrAfter and RefPicSetStFoll are already marked as “used forshort-term reference”.

3. All pictures in the DPB that are not included the RPS are marked as“unused for reference”. I.e. all reference pictures in the decodedpicture buffer that are not included in RefPicSetLtCurr,RefPicSetLtFoll, RefPicSetStCurrBefore, RefPicSetStCurrAfter orRefPicSetStFoll are marked as “unused for reference”.

4. A reference picture list is created at the decoder which contains thereference pictures from the RefPicSetLtCurr, RefPicSetLtFoll,RefPicSetStCurrBefore, RefPicSetStCurrAfter or RefPicSetStFoll. Thecurrent picture is decoded using the reference pictures of the referencepicture list.

Another way to formulate the steps performed in the decoder according tothe embodiments is:

1. The decoder decodes the RPS of the current picture (before decodingthe picture)

2. If there is a long-term reference picture signaled in the RPS withPOC=X and there is no picture in the DPB marked as “used for long-termreference” with POC=X and there is a picture with POC=X in the DPBmarked “used for short-term reference” that picture is marked as “usedfor long-term reference”.

3. All POCs of the pictures in the DPB that are not included the RPS aremarked as “unused for reference”.

4. The current picture is decoded.

5. The current picture is marked as “used for short-term reference”,Hence each decoded picture is marked as a short-term reference pictureimmediately after it is decoded. If the decoded picture should be along-term reference picture, it should be marked as a long-termreference picture in the very next picture. The picture will then neveractually be used as short-term since marking (in this case marking tolong-term) takes place before actual picture decoding of the nextpicture.

The POCs of the pictures that are in the RPS but have no correspondingpicture in the DPB marked as “used for short-term reference” or “usedfor long-term reference” may be inferred as lost pictures depending onother syntax elements i.e. the used_by_curr_pic flag.

As illustrated in the flowchart of FIG. 2, a method performed by adecoder for managing reference pictures to be used for decoding anencoded representation of a picture of a video sequence bitstream isprovided according to an embodiment.

A reference picture set received from an encoder is decoded 201, whereinthe reference picture set comprises information of the referencepictures to be kept in a decoded picture buffer. A POC value indicatedin the reference picture set is decoded 202 and it is determined 203 ifthe POC value indicated in the reference picture set corresponds to ashort-term reference picture or a long-term reference picture. If thePOC value indicated in the reference picture set corresponds to along-term reference picture, 203 a long-term reference picture is kept206 in a decoded picture buffer as a long-term reference picture ifthere is a long-term reference picture in the decoded picture buffer 205with a POC value corresponding to the POC value indicated in thereference picture set.

If the POC value indicated in the reference picture set corresponds to along-term reference picture 203, a short-term reference picture ismarked 207 in the decoded picture buffer as a long-term referencepicture and kept in the decoded picture buffer if there is a short-termreference picture in the decoded picture buffer 205 with a POC valuecorresponding to the POC value indicated in the reference picture set.

If the POC value indicated in the reference picture set corresponds to ashort-term reference picture, 203 a short-term reference picture is kept208 in the decoded picture buffer as a short-term reference picture ifthere is a short-term reference picture in the decoded picture bufferwith a POC value corresponding to the POC value indicated in thereference picture set.

Long-term reference pictures and short-term reference pictures are thesame kind of pictures with the only difference that they are markeddifferently in the DPB; “used for long-term reference” and “used forshort-term reference”, correspondingly. The reason to have this markingis

1) to be able to signal them more efficiently (short-term pictures canbe encoded with variable length codes while long-term ref pictures canbe encoded with fixed length codes).

2) to handle them differently in the decoding process (e.g. motionvectors from short-term pictures may be scaled in motion vector scalingwhile motion vectors from long-term pictures are not).

There does not have to be a distinct difference in POC steps betweenshort-term and long-term, i.e. can be as close to the current picture as1 POC step away. However, one alternative is to determine that long termpictures must have negative DiffPOC compared to the current picture i.e.precede it in output order.

The methods according to the embodiments can be implemented by acomputer program product encoded with computer program code means which,when loaded and executed by a processor, cause performance of the methodaccording to the embodiments.

A computer readable storage medium encoded with instructions which, whenloaded and executed by a processor, cause performance of the methodaccording to the embodiments is also provided.

In one alternative there is a restriction that reference pictures thathave been in the DPB for more than a specific period of time, whichpreferably is expressed in POC-steps, must be signaled as long-termpictures in the Buffer Descriptions.

In one alternative there is a restriction that reference pictures thathave been displayed (outputted) cannot be included as short-termreference pictures in a Buffer Description of a current picture if theirDiffPOC( ) compared to the current picture is positive.

In one alternative there is a syntax element, e.g. a flag or an id, sentfor each picture e.g. in the slice header to indicate if the currentpicture can be used as a long-term picture or not. Preferably there is arestriction that a picture A can only indicate that it may be used forlong-term reference if there is no long-term reference picture in theDPB with the same POC when A is decoded. In another alternative, thereference picture list construction allows for two pictures with thesame POC, where one picture is a long-term picture and the other pictureis a short-term picture. It is preferred that short-term pictures comebefore long-term pictures in the reference picture list.

The embodiments of the invention may be applied to any suitable videocodec comprising the encoder and/or the decoder according to theembodiments.

As mentioned above, the mechanisms described above are done both in theencoder as well as in the decoder. The encoder and the decoder,respectively, comprises a processor configured to perform the functionsaccording to the above described embodiments. Further the encoder andthe decoder, respectively comprises a memory for storing e.g. RPS,decoded pictures, and other picture information such as referencepictures in reference picture lists. The memory may also compriseinstructions to be executed on the processor such that the processor canperform the steps according to the embodiments.

FIG. 3 schematically illustrates an encoder in the context of theembodiments of the present invention.

Accordingly, an encoder, comprising one or more processors and e.g. oneor more memories, is configured to carry out the methods according tothe embodiments is provided.

FIG. 3 is a schematic diagram showing some components of the encoder300. The encoder comprises a processor 302. The processor 302 could beany combination of one or more of a suitable central processing unit(CPU), multiprocessor, microcontroller, digital signal processor (DSP),application specific integrated circuit (ASIC) etc., capable ofexecuting software instructions contained in a computer program storedin one or more memories 301.

Thus, an encoder 300 configured to encode a representation of a currentpicture of a video sequence of multiple pictures using referencepictures is illustrated in FIG. 3. The encoder comprises a processor 302configured to assign a POC value for the current picture, wherein thePOC value is assigned such that the decoder can perform the methodaccording to the embodiments of the present invention. The processor 302is configured to include the assigned POC value in a reference pictureset to be sent to a decoder, wherein the reference picture set comprisesinformation of the reference pictures to be kept in a decoded picturebuffer of the decoder. Hence, the encoder is configured to encode an RPSincluding all short-term reference pictures and long-term referencepictures that are used by the current picture or may be used by picturesfollowing the current picture and to encode the current picture usingall or a subset of the reference pictures included in the RPS.

FIG. 4 schematically illustrates a decoder according to an embodiment ofthe present invention.

Accordingly, a decoder, comprising one or more processors and e.g. oneor more memories, is configured to carry out the methods according tothe embodiments is provided.

FIG. 4 is a schematic diagram showing some components of the decoder400. The decoder comprises a processor 402. The processor 402 could beany combination of one or more of a suitable central processing unit(CPU), multiprocessor, microcontroller, digital signal processor (DSP),application specific integrated circuit (ASIC) etc., capable ofexecuting software instructions contained in a computer program storedin one or more memories 401.

A decoder 400 for managing reference pictures to be used for decoding anencoded representation of a picture of a video sequence is providedaccording to one embodiment. The decoder is configured to decode areference picture set received from an encoder, wherein the referencepicture set comprises information of the reference pictures to be keptin a decoded picture buffer and to decode a POC value indicated in thereference picture set. The decoder comprises a processor 402 configuredto determine if the POC value indicated in the reference picture setcorresponds to a short-term reference picture or a long-term referencepicture. The processor 402 is configured to keep a long-term referencepicture in a decoded picture buffer as a long-term reference picturewhen there is a long-term reference picture in the decoded picturebuffer with a POC value corresponding to the POC value indicated in thereference picture set and when the POC value indicated in the referencepicture set corresponds to a long-term reference picture. The processor402 is further configured to mark a short-term reference picture in thedecoded picture buffer as a long-term reference picture and keeping itin decoded picture buffer when there is a short-term reference picturein the decoded picture buffer with a POC value corresponding to the POCvalue indicated in the reference picture set and when the POC valueindicated in the reference picture set corresponds to a long-termreference picture. In addition, the processor 402 is further configuredto keep a short-term reference picture in the decoded picture buffer asa short-term reference picture when there is a short-term referencepicture in the decoded picture buffer with a POC value corresponding tothe POC value indicated in the reference picture set and when the POCvalue indicated in the reference picture set corresponds to a short-termreference picture.

According to an embodiment the processor is configured to determine ifthe POC value indicated in the reference picture set corresponds to ashort-term reference picture or a long-term reference picture bydetermining if the POC value indicated in the reference picture set isincluded in one of the lists PocLtCurr or PocLtFoll in which case thePOC value corresponds to a long-term reference picture or if the POCvalue indicated in the reference picture set is included in one of thelists PocStCurrBefore, PocStCurrAfter or PocStFoll in which case the POCvalue corresponds to a short-term reference picture.

Moreover, the long-term reference pictures in the reference picture setmay be handled before the short-term reference pictures to mark thereference pictures in the decoded picture buffer.

The encoder may be an HEVC encoder and the decoder may be an HEVCdecoder, but the embodiments are not limited to HEVC. The encoder andthe decoder, respectively may be implemented in a mobile device or inany type of video camera and/or display.

What is claimed is:
 1. An encoder configured to: encode, into a sliceheader portion of a bitstream, a plurality of Picture Order Count (POC)values; for each POC value of the plurality of POC values: i) if the POCvalue indicates a long-term reference picture and there is a referencepicture in a decoded picture buffer (DPB) with a POC value that matchesthe POC value: if the matching reference picture in the DPB is marked asa long-term reference picture, then include the matching referencepicture in the DPB in a long-term reference picture set; and if thematching reference picture in the DPB is marked as a short-termreference picture, then mark the matching reference picture in the DPBas a long-term reference picture, and include the matching referencepicture in the DPB in the long-term reference picture set; and ii)otherwise, if the POC value indicates a short-term reference picture andthere is a reference picture in the DPB with a POC value that matchesthe POC value: include the matching short-term reference picture in theDPB in a short-term reference picture set; and mark all pictures in theDPB that are not included in either the long-term reference picture setor the short-term reference picture set as unused for reference.
 2. Theencoder of claim 1, wherein: the POC value indicates a long-termreference picture if the POC value is included in one of PocLtCurr orPocLtFoll; or the POC value indicates a short-term reference picture ifthe POC value is included in one of PocStCurrBefore, PocStCurrAfter, orPocStFoll.
 3. The encoder of claim 1, wherein the encoder is configuredto, if the POC value indicates a long-term reference picture, determineif the matching reference picture in the DPB is a long-term referencepicture before determining if the matching reference picture in the DPBis a short-term reference picture.
 4. The encoder of claim 1, whereinthe encoder comprises a processor and a memory for storing instructionsto be executed on the processor such that the processor performs atleast the encoding, into the slice header portion of the bitstream, theplurality of POC values.
 5. The encoder of claim 1, wherein the encoderis implemented in a device which comprises one or more of: a mobiledevice, a video camera, or a display.
 6. An encoding method comprisingthe steps of: encoding, into a slice header portion of a bitstream, aplurality of Picture Order Count (POC) values; for each POC value of theplurality of POC values: i) if the POC value indicates a long-termreference picture and there is a reference picture in a decoded picturebuffer (DPB) with a POC value that matches the POC value: if thematching reference picture in the DPB is marked as a long-term referencepicture, then include the matching reference picture in the DPB in along-term reference picture set; and if the matching reference picturein the DPB is marked as a short-term reference picture, then mark thematching reference picture in the DPB as a long-term reference picture,and include the matching reference picture in the DPB in a long-termreference picture set; and ii) if the POC value indicates a short-termreference picture and there is a reference picture in the DPB with a POCvalue that matches the POC value: include the matching short-termreference picture in the DPB in a short-term reference picture set; andmarking all pictures in the DPB that are not included in either thelong-term reference picture set or the short-term reference picture setas unused for reference.
 7. The encoding method of claim 6, wherein: thePOC value indicates a long-term reference picture if the POC value isincluded in one of PocLtCurr or PocLtFoll; or the POC value indicates ashort-term reference picture if the POC value is included in one ofPocStCurrBefore, PocStCurrAfter, or PocStFoll.
 8. The encoding method ofclaim 6, further comprising: if the POC value indicates a long-termreference picture, determining if the matching reference picture in theDPB is a long-term reference picture before determining if the matchingreference picture in the DPB is a short-term reference picture.
 9. Theencoding method of claim 6, wherein a processor performs the encodingsteps and a memory stores instructions to be executed on the processorsuch that the processor can perform at least the encoding steps.
 10. Theencoding method of claim 6, wherein the method is implemented on adevice which comprises one or more of: a mobile device, a video camera,or a display.
 11. A non-transitory computer readable medium storinginstructions that, when executed by a processor, cause the processor toperform encoding operations comprising: encoding, into a slice headerportion of a bitstream, a plurality of Picture Order Count (POC) values;for each POC value of the plurality of POC values: i) if the POC valueindicates a long-term reference picture and there is a reference picturein a decoded picture buffer (DPB) with a POC value that matches the POCvalue: if the matching reference picture in the DPB is marked as along-term reference picture, then including the matching referencepicture in the DPB in a long-term reference picture set; and if thematching reference picture in the DPB is marked as a short-termreference picture, then marking the matching reference picture in theDPB as a long-term reference picture, and including the matchingreference picture in the DPB in a long-term reference picture set; andii) if the POC value indicates a short-term reference picture and thereis a reference picture in the DPB with a POC value that matches the POCvalue: including the matching short-term reference picture in the DPB ina short-term reference picture set; and marking all pictures in the DPBthat are not included in either the long-term reference picture set orthe short-term reference picture set as unused for reference.
 12. Thenon-transitory computer readable medium of claim 11, wherein: the POCvalue indicates a long-term reference picture if the POC value isincluded in one of PocLtCurr or PocLtFoll; or p1 the POC value indicatesa short-term reference picture if the POC value is included in one ofPocStCurrBefore, PocStCurrAfter, or PocStFoll.
 13. The non-transitorycomputer readable medium of claim 11, wherein the instructions that,when executed by a processor, cause the processor to perform encodingoperations further comprising: if the POC value indicates a long-termreference picture, determining if the matching reference picture in theDPB is a long-term reference picture before determining if the matchingreference picture in the DPB is a short-term reference picture.
 14. Thenon-transitory computer readable medium of claim 11, wherein thenon-transitory computer readable medium forms part of a device whichcomprises one or more of: a mobile device, a video camera, or a display.